This application claims the benefit of a Japanese Patent Application No. 11-305233 filed Oct. 27, 1999, in the Japanese Patent Office, the disclosure of which is hereby incorporated by reference.
1. Field of the Invention
The present invention generally relates to image pickup apparatuses and image processing methods, and more particularly to an image pickup apparatus which picks up an image in divisions and to an image processing method which processes the image which is picked up in divisions.
2. Description of the Related Art
Digital cameras have become increasingly popular. The increasing popularity of the digital camera was gained by the ease with which the image can be processed as digital information, matching the recent trend of processing various kinds of data electronically. In addition, it would be extremely troublesome to convert the image which is picked up by a conventional camera using a silver film into the digital information.
The digital camera can be used in various applications to bring out the advantageous features thereof. However, the resolution of the image which is picked up by the digital camera is not as high compared to that picked up by the conventional camera using the silver film. Although recently, the number of pixels of a CCD image pickup element used in the digital camera has increased, it is still not large enough for picking up extremely fine characters and patterns with a satisfactory reproducibility.
Accordingly, a method has been proposed to obtain partial images by picking up parts of a target object with a narrow field angle, and combining the partial images to generate a full image of the target object, which is as if the image were picked up by a high-density image pickup element having a large number of pixels with a wide field angle. When generating a high definition image by this proposed method, the following points are very important when picking up the partial images.
First, each part of the target object must be included in one of the partial images. If a certain part of the target object is missing from all of the partial images, the full image which is generated by combining the partial images will naturally be missing this certain part. Second, each partial image must have a region which overlaps an adjacent partial image. The partial images are combined based on image information of such overlapping regions. Generally, the accuracy of the image combining process improves as the overlapping regions become larger, but the number of partial images to be picked up consequently increases, to thereby increase the time required to carry out the image combining process.
Accordingly, when picking up the partial images of the target object, each part of the target object must be picked up, and in addition, the overlapping regions must be provided in an appropriate number of partial images. However, such an operation requires a skilled operator to perform, and furthermore, such an operation is extremely troublesome to perform. In view of the above, various methods have been proposed to pick up the partial images that will not put such a burden on the operator who operates the image pickup apparatus.
For example, a Japanese Laid-Open Patent Application No. 11-75108 proposes a method which facilitates generation of a combined image 190 shown in FIG. 1A by overlapping a partial image 191 shown in FIG. 1B which is already picked up and a partial image (through-image) 195 shown in FIG. 1C which is to be picked up or, time-divisionally displaying an image 199 shown in FIG. 1D. By displaying the partial images 191 and 195 on a monitor by overlapping a right end portion A of the partial image 191 and a left end portion A′ of the partial image 195, the operator adjusts an image pickup range (through-image) of a camera so that the overlapping portions A and A′ match, and records the combined image.
On the other hand, a Japanese Laid-Open Patent Application No. 10-186551 proposes a method which compares a portion of the partial image which is already picked up and a portion of the present through-image, and records the through-image as one of the partial images if the two compared images match. In addition, this proposed method picks up the full image of the target object, displays the position of the present through-image on the full image, and also displays the position of the partial image which is already picked up on the full image. As a result, this proposed method assists the operation to pick up the partial images so that the overlapping regions of the partial images positively exist and no missing partial image of the target object exists.
However, according to the method proposed in the Japanese Laid-Open Patent Application No. 11-75108, the overlapping regions of the partial images 191 and 195 are displayed, and for this reason, it is difficult to detect a characterizing feature in the overlapping regions if similar image information continues in the vicinity of the overlapping regions, as described below in conjunction with FIGS. 2A through 2C.
For example, it is assumed for the sake of convenience that the image of a target object 1 shown in FIG. 2A is picked up in two divisions, namely, a left partial image 2 shown in FIG. 2B corresponding to the left part of the target object 1 and a right partial image shown in FIG. 2C corresponding to the right part of the target object 1. A right portion B of the left partial image 2 is displayed in an overlapping manner on a left portion C of the right partial image 3 on the display. However, the image information in the right portion B is substantially the same in the horizontal direction for virtually the entire right portion B, and a position in the right portion B cannot be specified. As a result, it is extremely difficult for the operator to accurately overlap the left and right partial images 2 and 3 so that the right and left portions B and C perfectly match.
On the other hand, when combining the partial images arranged in one direction, as in the case of a panoramic image, a predetermined one of right, left, top and bottom end portions of the partial images is automatically displayed. But in a case where the image of the target object is picked up in four divisions, namely, a top right partial image 4, a bottom left partial image 5, a bottom right partial image 6 and a top right partial image 7 shown in FIG. 3, for example, it is extremely difficult to automatically judge the overlapping end portions of the partial images 4 through 7. For example, if the partial image 4 is picked up first, and the end portion of this partial image 4 is to be displayed in an overlapping manner on the end portion of the partial image (through-image) which is to be picked up next, the end portion to be overlapped differs depending on whether the partial image 5 or the partial image 7 is to be picked up next. It is extremely difficult to automatically judge the end portion to be overlapped, and it is also extremely troublesome to manually specify the end portion to be overlapped.
Furthermore, when displaying not only a portion of the image but all of the partial images which are picked up in an overlapping manner, no operation is actually carried out to overlap the end portions. Hence, the operator must carry out a troublesome operation of picking up the through-image while carefully confirming the display on the monitor the correspondence of the left end portion of the through-image and the right end portion of the partial image which is already picked up, for example. In addition, since the display is made on the monitor by only focusing on the overlap between two partial images at two successive image pickup times, the image pickup times do not become consecutive when picking up three or more partial images, and it is impossible to confirm the overlap between the adjacent partial images displayed on the monitor. In other words, in the case of the target object shown in FIG. 3, if the partial images 4, 5, 6 and 7 are picked up in this order, it is impossible to confirm the overlap of the partial images 4 and 7, because the image pickup times of these partial images 4 and 7 are not consecutive. Therefore, the method proposed in the Japanese Laid-Open Patent Application No. 11-75108 is not suited for a case where the image of the target object is picked up in divisions such that the adjacent partial images do not become continuous in time.
On the other hand, according to the method proposed in the Japanese Laid-Open Patent Application No. 10-186551, the problems of the method proposed in the Japanese Laid-Open Patent Application No. 11-75108 are unlikely to occur, since the overlap of the partial images is judged automatically. However, it is necessary to confirm the overlap of the through-image with respect to all of the partial images which are already picked up, and an extremely large amount of data processing is required for this confirmation. As a result, the required processing time increases, and the cost of the image pickup apparatus also increases. Moreover, there is a limit in the existing processing accuracy with which the overlap of the through-image and the partial image which is already picked up is confirmed, and the reliability of this confirmation process is not very high. Furthermore, there is a high possibility that a part of the image of the target object will not be picked up, since it is impossible to know which partial images have already been picked up.